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Education Needs to Further Examine Data Collection on English Language Learners in 
Charter Schools 


Over the past decade, the number of students who are English Language Learners (ELLs) 
enrolled in U.S. public schools has grown substantially. At the same time, charter schools have 
expanded the educational options for ELLs and their families. As the number of charter schools 
continues to grow, questions have arisen about how charter schools—a relatively new 
phenomenon in public education that began in the 1990s—are serving special student 
populations, including ELLs. Following the completion of our study comparing the enrollment of 
students with disabilities in charter schools and traditional public schools, GAO began a similar 
study evaluating the enrollment of ELLs in charter schools and traditional public schools.’ As 
part of this study, we obtained a dataset from the U.S. Department of Education (Education) 
containing school-level data that could be used to compare ELL enrollment in charter schools to 
ELL enrollment in traditional public schools for school year 2010-11. This was the only dataset 
available at the time that could be used for our analysis. Further, while this dataset was 
designed to contain enrollment data only on ELLs in ELL programs, Education officials initially 
told us there was no reason to expect that charter schools would be less likely than other 
schools to report data for the dataset. In accordance with Government Auditing Standards and 
GAO policy, we assessed the quality of the ELL data as part of our study and found problems 
with the data’s quality that prevented us from completing the study.” 


This letter frames our findings concerning Education’s data on ELL enrollment around two 
questions: (1) What is the quality of Education’s data on ELLs in charter schools? and (2) What 
efforts has Education taken to improve its collection of ELL data from charter schools? We 
conducted this review from November 2012 to July 2013 in accordance with generally accepted 
government auditing standards. See enclosure | for details on our scope and methodology. 


1GAO, Charter Schools: Additional Federal Attention Needed to Help Protect Access for Students with Disabilities, 
GAO-12-543 (Washington, D.C.: June 7, 2012). 


2See section 6.65 of Government Auditing Standards, 2011 Revision, GAO-12-331G (Washington, D.C-.: 
December 2011). 


Results in Brief 


GAO was unable to compare ELL enrollment in charter schools to ELL enrollment in traditional 
public schools because Education’s only available data on school-level ELL enrollment were 
unreliable and incomplete. Specifically, for over one-third of charter schools, the field for 
reporting the counts of ELLs enrolled in ELL programs was left blank. These blank fields cannot 
reliably be interpreted to mean that the charter schools did not have ELLs enrolled. Education 
officials told us that these school-level ELL data do not have a data steward (that is, an office 
responsible for overseeing the quality of the data) and that the quality of the data is not 
examined on a regular basis. A definitional issue may have resulted in states excluding some 
students from the reported counts, and some charter schools may have failed to submit required 
data to their states. Education officials said they had not systematically studied charter school 
nonreporting but acknowledged that states sometimes have difficulty obtaining data from charter 
schools. Some charter schools with blank ELL counts were also missing data from other 
important datasets, such as those on school performance, which suggests potential broader 
problems with charter school reporting. 


Education has a new ELL school-level data collection effort underway that will address the 
problem with the definition used in the current school-level ELL dataset. However, Education 
has not articulated how it will address the broader problem with charter school nonreporting that 
prevented us from completing our study. Further, Education plans to stop collecting school-level 
data for the ELL dataset with the problematic definition, but it plans to continue collecting these 
data for the local educational agency? (LEA) and state levels. 


GAO is recommending that Education systemically evaluate the extent of charter school 
nonreporting across other data sets and explore whether collection of the poorly defined data on 
ELLs enrolled in ELL programs can be phased out at the LEA and state levels, if the data quality 
problems are not addressed. 


In its comments on a draft of this report, Education questioned why data issues prevented GAO 
from completing its study and stated that GAO could have proceeded with its study by focusing 
on the factors affecting patterns of ELL enrollment in charter schools and traditional schools and 
the techniques charter schools use to outreach to ELLs. We disagree with Education because 
without information showing what the patterns of ELL enrollment in charter schools actually are, 
a study of the factors affecting those patterns would not be practicable, useful or informative. 
With respect to our recommendations, Education did not express agreement or disagreement. 
Education said that it was aware of problems with charter school reporting prior to the GAO 
study and that it has efforts already underway to address charter school nonreporting, including 
developing a complete directory of all charter schools in operation. We believe those efforts, 
while necessary, do not fully address the problems with charter school nonreporting that we 
found or illuminate the reasons for nonreporting of charter school data. Education also said it is 
exploring options regarding the LEA- and SEA-level counts, including whether these data can 
be phased out if the data quality problems are not addressed. 


3Generally, local educational agencies are school districts. 
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Background 


By definition, ELLs come from a non-English speaking background and have not yet developed 
the level of English proficiency needed for academic success. Various federal requirements 
pertain to ELLs, including the requirement under the Elementary and Secondary Education Act 
of 1965 (ESEA), as amended, for states to include ELLs in the academic assessments 
prescribed under the act and to report on their achievement.* Further, according to Education’s 
Office for Civil Rights, ELLs must be provided with alternative educational services until they are 
proficient enough in English to participate meaningfully in regular educational programs. 


In school year 2010-11, 40 states had at least one charter school operating in the state.° 
Relative to traditional schools, charter schools served only a small fraction of school age 
children—about 1.8 million (4 percent) of the 46 million public school students in those 40 states 
in school year 2010-11. Charter schools are public schools created to achieve a number of 
goals, including encouraging innovation in public education and addressing failing schools. 
Generally, charter schools operate with more autonomy than traditional public schools but are 
subject to various federal requirements in the same way as traditional public schools, including 
requirements under ESEA, civil rights laws, and the Individuals with Disabilities Education Act. 


As part of its mission and oversight, Education collects and reports a wide array of data on 
public schools, such as student enrollment, demographics, and academic performance. 
Education also publishes these data in several reports, most notably its Digest of Education 
Statistics and The Condition of Education that are used by researchers who follow trends in 
education. While Education often reports these data for each state or the nation as a whole, the 
data originate from individual schools. As shown in figure 1, generally, data, such as ELL 
counts, are reported from schools through LEAs and state educational agencies (SEAs) to 
Education. SEAs submit the data to Education, but, to do this, SEAs must first collect the data 
from schools and LEAs. 


490 U.S.C. §§ 6311(b)(3)(C)(ix)(II) and 6311(h)(1)(C)(i). 
5The District of Columbia is included as one of the 40 states. 
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Figure 1: Flow of Data from Schools to Education 
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Source: GAO. 


Depending on the requirements for each dataset, SEAs may report data to Education at the 
school-level (that is, broken down by each individual school) or aggregated to the LEA- or SEA- 
level.© Having school-level data is important to researchers wishing to compare charter schools 
to traditional schools because once data are aggregated to the LEA or SEA levels, the ability to 
separate out charter schools from traditional schools is lost. That is why our study comparing 
enrollment of ELLs in charter schools and traditional schools required school-level data. The 
only comprehensive school-level data on ELL counts that Education had at the time of our study 
was of ELLs “enrolled in English language instruction educational programs” designed for 
ELLs.’ Education periodically collects school-level counts of ELLs in its Civil Rights Data 
Collection (CRDC), but, at the time of our review, the recent CRDC data only covered a sample 
of schools that was not designed to be statistically representative of charter schools.® Education 
collects other data on numbers of ELLs at the LEA and SEA levels, but not at the school-level. 
These include data on the number of 


e ELLs enrolled,? 


e ELLs served by the Title III program, '? and 


®To lessen the reporting burden on SEAs, for many datasets, Education does not require school-level data and 
instead requires only LEA- or SEA-level data. Requiring school-level data would greatly increase the quantity of data 
that SEAs must submit because there are many more schools than LEAs and SEAs. 


7In Education's datasets, the term “limited English proficient” (LEP) is used to refer to ELLs. The school-level dataset 
we used was counts of “LEP students enrolled in English language instruction educational programs designed for 
LEP students,” referred to as data group 123. 


®The CRDC is conducted approximately every other school year and usually covers only a sample of schools. 


Education is in the process of collecting school year 2011-12 CRDC data for all schools, and Education officials 
estimated that these data will be available for use in late 2013 or early 2014. 


°This dataset is of limited English proficient students “enrolled in an elementary or secondary school at any time 
during the school year” and is referred to as data group 678. 


1The Title III program provides grants to SEAs to improve the education of ELLs by helping them learn English and 
meet challenging state academic content and student academic achievement standards. Title Ill program grants are 
authorized by Title Ill of ESEA. 
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e former ELLs who have achieved proficiency in English and are no longer receiving Title 
Ill services. 


Unlike traditional public schools that are part of a LEA, some states establish charter schools as 
their own LEA. Other states require charter schools to be part of a larger LEA, while still others 
allow charter schools the option to choose between being a distinct LEA or part of a larger 
LEA.'’ When a charter school is part of a larger LEA, its data are supposed to be included along 
with data from other schools in the LEA’s totals reported to Education. Conversely, a charter 
school that is its own LEA would report only a single school’s data even when reporting at the 
LEA-level. ‘2 


Nonreporting and Definitional Issues Undermine the Quality of Education’s Data on ELLs 
in Charter Schools 


We were unable to compare ELL enrollment in charter schools to ELL enrollment in traditional 
public schools nationwide or by state because of problems with Education’s only available 
dataset of school-level ELL enrollment counts. Specifically, in our review of this dataset, we 
found that a substantial percentage of charter schools—37 percent in school year 2010-11— 
had blanks in the field capturing ELL enrollment counts.'* While some of these schools may not 
have any ELL students, our analyses suggested that some schools that have ELLs enrolled did 
not report them. ‘'* Further, the percentage of schools with blanks in the field for ELL counts 
varied by state. In 14 states, at least 60 percent of the charter schools had blank fields. In two 
states generally known to have large numbers of ELLs, New York and New Jersey, over 80 
percent of charter schools had blank fields.*® In contrast, in six states all charter schools had 
numerical values—including zeros—for ELL counts; that is, no charter schools had blank fields 
for their ELL counts in these states. Figure 2 shows the distribution of charter schools with blank 
ELL counts in the 40 states with charter schools. 


1In addition, some states allow charter schools to be their own LEA for some purposes and part of a larger LEA for 
others, including for purposes of special education. 


12Some charter school LEAs comprise more than one charter school. In such cases, the LEA would submit data for 
all charter schools within the LEA. 


13We found that 21 percent of traditional public schools had blank ELL enrollment counts. However, the analyses that 
we and Education conducted on schools with blank ELL counts focused on charter schools because their higher 
percentage of blanks (37 percent) would likely hinder our ability to make meaningful comparisons between charter 
schools and traditional public schools. 


141n this report, we refer to schools as leaving fields blank in Education’s dataset. As shown in figure 1, school data 
originate with schools but are reported to Education by SEAs. Because of the SEA’s role of compiling and submitting 
data, it is possible that some SEAs received ELL counts for schools that they did not to submit to Education. 


S according to data from the American Community Survey, New York and New Jersey had the 3 and 7™ highest 


numbers of ELLs, respectively, among the 50 states based on 2009 3-year estimates. Specifically, these ACS data 
are on the number of individuals ages 5 to 21 who speak a language other than English at home and speak English 
less than very well. 
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Figure 2: Percentage of Charter Schools With Blank ELL Enrollment Counts in School 
Year 2010-11, by State 
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Number of states 


Source: GAO analysis of Education’s school-level ELL enrollment data for SY2010-11. 


“States with less than 5 charter schools operating in school year 2010-11. 


While Education’s data reporting protocols for this dataset allow blank fields if the ELL counts 
are zero, these blanks cannot reliably be interpreted as zeros, especially given the large number 
of charter schools with blank ELL counts. In response to concerns we raised about the data, 
Education conducted analyses that found some charter schools with blank ELL counts did, in 
fact, have ELLs enrolled. Specifically, Education found that some of these charter schools were 
their own LEA and had LEA-level counts for ELLs in a different dataset which is collected at the 
LEA-level but not at the school-level. Also, some of the charter schools with blank ELL counts in 
the dataset we reviewed had reported assessment results for ELLs through another Education 
data collection. 


The difficulty of interpreting blank ELL counts in Education’s dataset is further complicated by 
how Education defines a key term for the dataset. Specifically, this dataset is a count of ELLs 
enrolled in “English language instruction educational programs,” not a simple count of all ELLs. 
When we began our study, Education officials told us there was no reason to expect that certain 
types of schools such as, charter schools or schools without Title Ill funding, would be less likely 
than other schools to report data for the dataset. However, after we identified the high 
percentage of charter schools with blank ELL counts, Education officials told us that the 
definition of “program” for the dataset is broad and left up to state interpretation. Specifically, 
they said that charter schools may be less likely to consider their ELLs as participating in a 
program, as defined by their state and, therefore, may not report their ELLs in this dataset. 
Further, they said that, in some states, the SEA may decide not to report ELLs for some schools 
if it determines that the schools did not have programs that met the state’s definition of 
“program” for the dataset. An Education official told us that officials in one state said they 
consider the term “program” for this dataset to mean the Title III program that provides grants to 
states to support the education of ELLs, and the state only reports ELLs in schools that receive 
Title Ill funding. This definitional issue could affect the reliability of ELL counts across all states, 
including the six states where no charter schools had blank ELL counts. 
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An additional likely cause for blank ELL counts in Education’s dataset is that some charter 
schools failed to submit required data to their state. Education officials said they had not 
systematically studied charter school nonreporting but acknowledged that states sometimes 
have difficulty obtaining data from charter schools. Further, Education officials told us that some 
state officials identified charter school nonreporting as a contributing factor to the large 
percentage of blank ELL counts in Education’s dataset. In addition, Education’s analyses 
suggest that some of the charter schools with no reported ELL data for the dataset we reviewed 
may have additional data reporting problems. Specifically, Education’s analyses found that 
some charter schools with blank ELL counts were also missing math or reading performance 
data or graduation rates for all students, including ELLs, which suggests these schools had 
broader reporting problems not limited to ELLs. In response to our questions about the 
problems with charter schools’ ELL data, Education officials told us that the school-level ELL 
data that we reviewed do not have a data steward (that is, an office responsible for overseeing 
the quality of the data) and that the quality of the data is not examined on a regular basis; 
instead, any quality reviews of the data are conducted on an ad hoc basis when the data are 
requested. 


The problems we identified with Education’s school-level ELL dataset coupled with Education’s 
own findings that key information was missing from other databases that collect information on 
charter schools, indicate potential broader problems with charter school reporting and 
undermine Education’s ability to have reliable, timely information to monitor charter schools. '® In 
addition, the problems GAO identified with the school-level ELL data appear to affect the quality 
of the LEA-level counts of ELLs enrolled in “English language instruction educational programs” 
that Education publishes and makes available to researchers in the Common Core of Data, the 
agency’s dataset that captures a broad range of information on education. Education officials 
told us that states submit school-level and LEA-level data separately to Education; therefore, 
the LEA-level data may not have the same problems that we identified with the school-level 
data. However, when we calculated and compared statewide ELL counts from two datasets— 
(1) the school-level dataset we received from Education and (2) LEA-level data from the 
Common Core of Data—we found that, for most states, the ELL counts for both datasets 
matched almost exactly, as shown in figure 3. This suggests that, even though states submit the 
school-level and LEA-level data separately to Education, both datasets are derived from the 
same source—data collected from schools. This further suggests that, as with the school-level 
data, the LEA-level data are incomplete and unreliable. For example, New Jersey’s ELL counts 
from its LEA-level and school-level data matched almost exactly even though over 80 percent of 
its charter schools did not have ELL counts in the school-level data. 


16Federal agencies are responsible for establishing and maintaining internal control to achieve their objectives such 
as enforcing compliance with federal laws and regulations, and having relevant, reliable, and timely information is a 
standard element of internal control. See OMB Circular A-123, “Management’s Responsibility for Internal Control”, 
revised December 21, 2004 and GAO, Standards for Internal Control in the Federal Government, GAO/AIMD-00- 
21.3.1 (Washington, D.C.: November 1999). 
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Figure 3: Percentage Difference between Numbers of English Language Learners 
Included in School-Level Dataset Analyzed by GAO and Local Educational Agency Level 
Data from Education’s Common Core of Data 
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Source: GAO analysis of states’ ELL counts in school year 2010-11; School-level dataset provided by Education and LEA-level data downloaded from Common Core of Data for states with charter schools. 


Note: California is excluded from the table because its ELL counts for school year 2010-11 were not included in the 
Common Core of Data. 


Education is Revising its ELL Data Collection Efforts, but Charter School Nonreporting 
Remains Problematic 


Beginning in school year 2013-14, Education plans to collect new school-level data on all ELLs 
regardless of whether or not they are enrolled in a “program,” thus addressing the issue with the 
term “program” that limits the usefulness of the current school-level dataset.'” According to 
Education officials, the new school-level data will, unlike the current school-level data, have a 
data steward and will receive regular data quality reviews. For the dataset that we analyzed, 
ELLs enrolled in English language instruction educational programs designed for ELLs, 
Education plans to halt collection of school-level data but continue collecting LEA- and SEA- 
level data. Given that the quality problems we found in the current school-level data appear to 
carry over to and reduce the usefulness of the LEA- and SEA-level data, the cost of continuing 
to collect these LEA- and SEA-level data may not be justified. 


In addition, Education officials are developing a workbook to serve as a guide for SEAs in 
reporting charter school data. Specifically, according to Education officials, the workbook will 
address issues that affect Education’s ability to maintain a complete and accurate directory of 
charter schools. For example, the workbook is intended to address issues, such as reporting on 
the opening and closing of charter schools and providing up-to-date information on charter 
schools’ LEAs and authorizers. Education plans to issue this workbook in the summer of 2013 
before the school year 2013-14 data collection cycle. However, Education has not articulated 
how it will address issues with charter school nonreporting that hindered our ability to compare 
ELL enrollment at charter schools to enrollment at traditional public schools. 


Conclusions 


The population of ELLs is growing and charter schools provide these students and their families 
an additional educational option. Therefore, it is important that data be available on the 


'7Currently, this dataset (referred to as data group 678) is collected only at the LEA and SEA levels. 
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characteristics of students enrolled in charter schools in order to assess how these schools are 
serving ELLs relative to traditional public schools. However, because of problems with the 
completeness and reliability of Education’s current school-level data, we could not determine 
how the enrollment of ELLs in charter schools and traditional public schools compare. Further, 
the nonreporting by charter schools that we identified and the subsequent finding by Education 
that some charter schools had not reported information in other datasets may signal a broader 
problem with charter school reporting. These data problems could have implications for 
Education’s ability to track key information related to charter schools and interfere with their 
ability to effectively oversee charter schools’ compliance with relevant federal requirements. 
While Education plans to begin collecting new school-level ELL data that does not have the 
definitional issue that characterizes its current data collection, the problem of nonreporting by 
charter schools remains. Further, the problems we identified with Education’s current school- 
level ELL data also appear to affect these data at the LEA and SEA levels. While Education has 
plans to stop collecting these data at the school-level, it plans to continue its LEA- and SEA- 
level data collection. The limited usefulness of the LEA- and SEA-level data may not be worth 
the cost of collecting the data, given the data quality problems we identified. 


Recommendations for Executive Action 
We recommend that the Secretary of Education take the following two actions: 


1. To more fully understand and address nonreporting of charter school data, conduct a 
systematic evaluation of other important data sets such as, datasets on academic 
achievement and graduates/completers to determine the extent of charter school 
nonreporting, assess underlying reasons for such nonreporting, and determine the steps 
needed to address nonreporting. 


2. Because the problems we identified with the school-level ELL data also affect the LEA- 
level data, explore whether the collection of the LEA- and SEA-level counts of ELLs 
enrolled in English language instruction educational programs can be phased out, if the 
data quality problems are not addressed. 


Agency Comments and Our Evaluation 


We provided a draft of this report to the U.S. Department of Education for review and comment. 
The U.S. Department of Education provided written comments which are reproduced in 
enclosure Il. 


In its comments, Education did not indicate agreement or disagreement with GAO’s 
recommendations. Education questioned why data issues prevented GAO from completing its 
study. Education posited that GAO could have proceeded with its study focusing on the factors 
affecting patterns of ELL enrollment in charter schools and traditional schools and the 
techniques charter schools use to outreach to ELLs. We disagree with Education. Lacking 
information showing what the patterns of ELL enrollment in charter schools actually are, a study 
of the factors that affect those patterns would not be practicable, useful or informative. Further, 
with respect to the data Education provided to GAO for its study, Education in its agency 
comments reported that they had already determined that the data were of “limited utility.” 
However, Education did not provide information on the problems with the data until after we 
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identified the high percentage of charter schools with blank fields in the dataset. As a result of 
our review of the data, Education conducted its own analyses and consulted with states to 
understand the potential causes for the problems. Education said that, other than the dataset 
that GAO used on “Limited English Proficient students enrolled in language instruction 
educational programs,” it has not had a school-level dataset on ELL enrollment data because, 
consistent with its program statutes, such a dataset has not been needed by its program offices 
to date. Moving forward, Education said that it will have two datasets with school-level ELL 
enrollment data and that one of these, the Civil Rights Data Collection for school year 2011-12, 
will be available for use by early 2014. Once these data are available, GAO will assess the 
quality and make a determination about whether the data can be used for a future study of ELL 
enrollment. 


In response to GAO’s first recommendation that Education conduct a systematic evaluation of 
other important datasets to determine the extent of charter school nonreporting, Education said 
that it was aware of problems with charter school reporting and that the department had efforts 
underway to improve the completeness and quality of charter school data prior to the GAO 
study. Specifically, Education said that it is developing a complete and accurate directory of 
charter schools and that this is a first step in ensuring high quality reporting because schools 
must be included in the directory before student counts can be reported for them. Education 
also stated that the lessons and outcomes from the directory review are informing its 
development of a charter school reporting workbook, which will assist states, along with 
traditional and charter school districts and authorizers, in reporting complete and high quality 
data. Further, Education said the technical assistance it provides to states and the attention it is 
giving to the charter school directory will encourage closer examination of the charter school 
data that states report to Education. We believe Education’s efforts to provide technical 
assistance to states, maintain an up-to-date directory of charter schools, and develop a charter 
school reporting workbook can help improve the collection of charter school data. However, 
these efforts are not designed to provide a clear picture of the extent of nor reasons for charter 
school nonreporting, and we believe a systemic evaluation could be useful for this purpose. 


In response to GAO’s second recommendation that Education explore whether the collection of 
LEA- and SEA-level counts for ELLs enrolled in “English language instruction educational 
programs’ can be phased out if the data quality problems are not addressed, Education said it is 
exploring how phasing out this data collection would meet the needs of various data users and 
impact the Common Core of Data, which is a compilation of information on all public schools 
and school districts in the United States. Further, its Office of Civil Rights and Title III office are 
discussing the option of providing instructions and other guidance to states and districts on the 
definition of “English language instruction educational programs” in the data collection in order 
to improve the quality and quantity of responses. Education said it plans to continue collecting 
these data at the LEA and SEA levels during this evaluative phase. 


Education also provided technical comments on the report, and we have made changes in the 
report, where appropriate, to address these comments. 


As agreed with your office, unless you publicly announce its contents earlier, we plan no further 
distribution of this report until 30 days from its issue date. At that time, we will send copies of 
this report to relevant congressional committees, the Secretary of Education, and other 
interested parties. In addition, the report will be made available at no charge on the GAO Web 
site at http://www.gao.gov. 
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If you or your staff have any questions about this report, please contact me at (202) 512-7215 or 
scottg@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs 
may be found on the last page of this correspondence. GAO staff who made key contributions 
to this report are Sherri Doughty, Assistant Director; Kathy Peyman, Analyst-in-Charge; Divya 
Bali; James Bennett; Holly Dye; Kirsten Lauber; Lise Levie; Ying Long; Luann Moy; and James 
Rebbe. 


Lhonge A Jeo 


George A. Scott 


Director 
Education, Workforce, and Income Security Issues 
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Enclosure I: Scope and Methodology 


We assessed the quality of Education’s enrollment data on English Language Learners (ELLs) 
in charter schools in order to determine if the data were reliable for the purpose of comparing 
ELL enrollment in charter schools to ELL enrollment in traditional public schools. We obtained 
from Education nationwide data for school year 2010-11 on all public schools, including charter 
schools. '® These data came from a dataset on the “unduplicated number of limited English 
proficient (LEP) students enrolled in English language instruction educational programs 
designed for LEP students.” (Education’s datasets use the term “limited English proficient” to 
refer to ELLs.) Based on an interview with Education officials and our review of available 
documentation about Education’s datasets, we concluded that this was the only available 
dataset that could be used for a valid comparison of ELL enrollment in charter schools to ELL 
enrollment in traditional public schools. Education had other datasets related to ELL enrollment, 
such as a dataset on the “unduplicated number of LEP students enrolled in an elementary or 
secondary school at any time during the school year.” However, these other datasets did not 
contain representative school-level data for charter schools that were needed for our review. 
Specifically, other datasets either (1) only included state-level or local educational agency (LEA) 
level data but not the school-level data needed for our analysis or (2) included school-level data 
for a sample of schools that was not designed to be statistically representative of charter 
schools. 


To determine the quality of Education’s ELL enrollment data, we conducted analyses and 
reviewed documentation such as data reporting protocols. Our analysis of the dataset included 
analyzing, at the national and state levels, the number of charter schools and traditional public 
schools that had a blank field for the number of ELLs enrolled. In addition, for some states with 
many charter schools with blank fields for ELL enrollment numbers, we reviewed the list of 
individual charter schools to assess the plausibility that the blanks indicated that no ELLs were 
enrolled in the schools. To do this, we considered evidence such as the names of charter 
schools and how their ELL count reporting compared to other schools in nearby zip codes. 
Further, we calculated statewide ELL counts from two datasets—(1) the school-level dataset we 
received from Education on the “limited English proficient (LEP) students enrolled in English 
language instruction educational programs designed for LEP students” and (2) LEA-level data 
from the Common Core of Data—and we compared these two statewide ELL counts to 
determine the extent to which they differed. Finally, we interviewed officials at Education about 
the data and about the analyses they conducted on the data reporting of charter schools that 
had blanks in the field for the number of ELLs enrolled. 


To determine the efforts Education has taken to improve its collection of ELL data from charter 
schools, we interviewed Education officials and reviewed relevant documents. 


We conducted this review from November 2012 to July 2013 in accordance with generally 
accepted government auditing standards. 


18Equcation also provided GAO with ELL enrollment data for school year 2009-10, but GAO’s detailed analysis, 
discussed in this letter, focused on the school year 2010-11 data because it was the most recent available data. 
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Enclosure Il: Agency Comments and Our Evaluation 


UNITED STATES DEPARTMENT OF EDUCATION 
OFFICE OF PLANNING, EVALUATION AND POLICY DEVELOPMENT 


ASSISTANT SECRETARY 
June 26, 2013 


Mr. George A. Scott 

Director 

Education, Workforce, and Income Security Issues 
Government Accountability Office 

Washington, DC 20548 


Dear Mr. Scott: 


Thank you for the opportunity to review and comment on the Government Accountability 
Office (GAO) draft report, Education Needs to Further Examine Data Collection on 
English Language Learners in Charter Schools (GAO-13-655R). The U.S. Department 
of Education (the Department) appreciates GAO’s interest in studying the distribution of 
English Language Learners (ELLs) in charter schools and traditional public schools, as it 
is critically important that school districts take steps to help ELL students overcome 
language barriers and to ensure that they can participate meaningfully in the district’s 
educational programs. We recognize, however, that GAO has concluded it is unable to 
complete the study at this time. Last July, GAO’s preliminary research approach 
included an outline to move beyond analysis of nationally representative data in order to 
explore factors affecting patterns of enrollment within charter schools and the techniques 
being used by charters to provide outreach to the traditionally underserved group of ELLs 
across our country. We believe that this additional analysis would have been useful to 
the Department and education field more broadly and could have been conducted at this 
time. 


GAO’s initial research approach started with analysis of existing data in order to 
“compare enrollment level of ELL students in charter schools and traditional public 
schools.”! A school-level dataset of ELL enrollment has not been part of the EDFacts 
collection because, consistent with our program statutes, this dataset has not been a need 
or requirement of Department program offices to date. Thus, we had not requested our 
grantees to take on this additional reporting burden. 


To help support GAO’s first phase of their study, however, the Department provided 
GAO with a dataset containing the most closely related set of school-level data available 
within EDFacts at the time--counts of “Limited English Proficient students enrolled in 
language instruction educational programs.” As discussed in subsequent conversations 
with GAO, these data are not equivalent to ELL student enrollment counts. Individual 
schools might not have programs that meet State definitions of a language instruction 
educational program (LIEP), but they may enroll ELL students whose needs are served 
through other means, such as supported inclusion within regular classroom settings. In 


‘GAO, ‘English Language Learners in Charter Schools (131193) Entrance Conference — July 31, 2012; 
11:00 a.m. ET’ agenda document. 
400 MARYLAND AVE. S.W., WASHINGTON, DC 20202 
www.ed.gov 
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these cases, the Department would not expect data to be reported for that individual 
school within EDFacts, since no students at the school meet the State’s definition of 
“Limited English Proficient students enrolled in language instruction educational 
programs.” 


It is important to note that the reason the term “program” in this data collection is left to 
State interpretation is that this is consistent with Title III program requirements. The 
Department cannot and does not dictate the type of LIEP States or LEAs are to provide 
(e.g., ESL (English as a second language), bilingual, etc.), either under Title III or under 
civil rights laws. Under Title VI of the Civil Rights Act and the Equal Educational 
Opportunities Act, all school districts, including charter schools that are LEAs and LEAs 
that do not receive Title III funds, must provide an alternative English language program 
that is recognized as sound by educational experts in the field, is fully implemented, and 
is shown to be successful. Case law and memoranda issued by the Department’s Office 
for Civil Rights (OCR) provide guidance for programs, but do not prescribe a particular 
type of program that a schoo! district must adopt to serve LEP students. School districts 
that receive Title III subgrants are required to provide services to ELLs that supplement 
the district’s basic English language program. We are hopeful that GAO comments about 
“data quality” or “completeness” will be viewed in this context and the context set forth 
below. 


We understand that in conducting this study, GAO was interested in the total number of 
ELLs enrolled in schools, not the number served by English language instruction 
programs. While a school-level dataset of total ELL enrollment is not currently collected 
or available for the reasons noted above, the Department will have two datasets with this 
information in the future. The Civil Rights Data Collection for school year (SY) 2011-12 
contains ELL enrollment data for all schools, and the Department anticipates that these 
data will be available for use by early 2014. In addition, the EDFacts data collection has 
already obtained OMB clearance to expand its SEA- and LEA-level collection of ELL 
enrollment counts to include school-level data starting in SY 2013-14. These datasets 
should facilitate further research into the enrollment of ELL students in charter schools 
and traditional public schools, and we are hopeful that these datasets will be useful to 
GAO and the public in the future. ; 


As currently drafted, the report includes the following recommendations for Executive 
action: 


Recommendation 1: To more fully understand and address nonreporting of charter 
school data, conduct a systematic evaluation of other important data sets such as, 
datasets on academic achievement and graduates/completers to determine the extent of 
charter school nonreporting, assess underlying reasons for such nonreporting, and 
determine the steps needed to address nonreporting. 


The Department already had efforts underway to improve the completeness and quality of 
the charter school data prior to GAO’s study. We have been aware of data concerns 
regarding charter schools, and we began addressing them in July 2011, with a systematic 
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review and reconciliation of directory data across three independent sources: 1) charter 
school directory data reported through EDFacts, 2) grant and subgrant award data from 
the Charter Schools Program, and 3) the National Charter School Resource Center’s 
authorizer survey. Maintaining a complete and accurate directory of charter schools is 
the first step to ensuring high quality reporting of such schools and the students they 
serve, since schools must first be included in the directory before additional student 
counts can be reported for them. 


The Department has taken additional steps to improve charter school inclusion in all 
aspects of EDFacts data collection as part of our overall commitment to ensuring a 
complete and accurate data collection across all schools in every State. The lessons and 
outcomes from the directory review are informing our development of a charter schools 
reporting workbook, which will assist States, along with traditional and charter school 
districts and authorizers, in reporting complete and high quality data. This workbook will 
be discussed during a session at the National Center for Education Statistics’ July 2013 
STATS-DC Data Conference. The session will emphasize the importance of reporting 
complete and accurate charter schools data and will provide technical assistance to the 
SEA staff responsible for EDFacts data submissions. We believe that our directed 
attention to the charter school directory, as well as technical assistance to SEAs through 
our EDFacts Partner Support Center, will encourage closer examination of the charter 
school data that SEAs are reporting to EDFacts. Additionally, we expect that the 
forthcoming workbook and conference session will foster more open communication 
between stakeholders who manage charter school data at the State, LEA, and school 
levels. 


Recommendation 2: Because the problems we identified with the school-level data also 
affect the LEA-level data, explore whether the collection of the LEA- and SEA-level 
counts of ELLs enrolled in English language instruction educational programs can be 
Phased out, if the data quality problems are not addressed. 


The Department continually reviews its inventory of reporting requirements, and remains 
open to modifying our reporting requirements when a more useful, and higher quality, 
alternative can be identified. We must continually weigh the value of alternatives that 
may produce more useful and higher quality data against the burdens the alternatives 
impose. Additionally, we routinely assess our regulatory data reporting requirements and 
seek to streamline them by maintaining items with high utility for supporting 
improvements in educational outcomes and accountability and eliminating items with 
high administrative costs and low utility. As discussed above, the EDFacts data 
collection has been approved by OMB to expand our SEA- and LEA-level collection of 
total ELL enrollment counts to include school-level data starting with SY 2013-14. The 
Department’s Office of Elementary and Secondary Education serves as the steward for 
the total ELL enrollment counts across all levels and conducts an annual review of the 
data submitted by States. 


Regarding the counts of “Limited English Proficient students enrolled in language 
instruction educational programs,” prior to this GAO study we had already determined 
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that EDFacts data on student participation in LIEP was of limited utility. For example, 
because the Title III program is limited by statute to overseeing SEA-level and LEA-level 
compliance, the school-level data is not utilized for Title III enforcement purposes. 
Therefore, the Department could no longer justify continuing the burden of collecting 
these school-level data in addition to the ELL enrollment counts being added at the 
school level for 2013-14. 


As we consider any changes to our data reporting requirements and data usage, we are 
taking the necessary steps to meet the needs of various data users. The Department is 
exploring what the impact would be upon the Common Core of Data (CCD) if reporting 
was switched from LEA-level counts of ELL students enrolled in language programs to a 
count of all ELL students enrolled, regardless of program participation. However, there 
are purposeful definitional differences in these items that may preclude the Department 
from simply eliminating the LEA and SEA collections as GAO suggests. Both the Title 
III program officials and the Department’s OCR have an interest in the data on students 
enrolled in English language programs as reported by States and school districts. These 
two offices are discussing the option of providing instructions and other guidance to 
States and districts on the definition of “English language instruction educational 
programs” in the data collection in order to improve the quality and quantity of 
responses. During this evaluative phase, it is important that the Department continue to 
collect both data groups in order to assess the likely impact of any change for all users. 


In addition to these comments, we have included an enclosure with technical points of 
clarification to the draft report, which I trust GAO will find useful. Please let us know if 
you have any questions about our comments in this letter or our clarification in the 
enclosure. Thank you for the opportunity to provide comments and clarification on the 
draft report. 


Sincerely, 


Ques I> 


Denise Forte 
Acting Assistant Secretary 


Enclosure — Technical Edits 


(131193) 
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This is a work of the U.S. government and is not subject to copyright protection in the 
United States. The published product may be reproduced and distributed in its entirety 
without further permission from GAO. However, because this work may contain 
copyrighted images or other material, permission from the copyright holder may be 
necessary if you wish to reproduce this material separately. 


GAO’s Mission The Government Accountability Office, the audit, evaluation, and 
investigative arm of Congress, exists to support Congress in meeting its 
constitutional responsibilities and to help improve the performance and 
accountability of the federal government for the American people. GAO 
examines the use of public funds; evaluates federal programs and 
policies; and provides analyses, recommendations, and other assistance 
to help Congress make informed oversight, policy, and funding decisions. 
GAO’s commitment to good government is reflected in its core values of 
accountability, integrity, and reliability. 


Qe 
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Obtaining Cop ies of cost is through GAO’s website (www.gao.gov). Each weekday afternoon, 
GAO Reports and GAO posts on its website newly released reports, testimony, and 
Testimony correspondence. To have GAO e-mail you a list of newly posted products, 


go to www.gao.gov and select “E-mail Updates.” 


Order by Phone The price of each GAO publication reflects GAO’s actual cost of 
production and distribution and depends on the number of pages in the 
publication and whether the publication is printed in color or black and 
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http:/Awww.gao.gov/ordering.htm. 


Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 
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